Papers with natural language models
Predicting Difficulty and Discrimination of Natural Language Questions (2022.acl-short)
Copied to clipboard
| Challenge: | Item Response Theory (IRT) has been used to numerically characterize question difficulty and discrimination for human subjects in domains including cognitive psychology and education. |
| Approach: | They explore the relationship between difficulty and discrimination in question-answering contexts by using IRT to characterize item difficulty and item discrimination. |
| Outcome: | The proposed models can predict difficulty and discrimination parameters for new questions and explain them with features of questions, answers, and associated contexts. |
Lexical Features Are More Vulnerable, Syntactic Features Have More Predictive Power (D19-55)
Copied to clipboard
| Challenge: | Existing metrics to quantify lexical diversity have been proposed. |
| Approach: | They propose to examine how generic language characteristics are impacted by text alterations. |
| Outcome: | The proposed models show that lexical features are more sensitive to text modifications than syntactic ones. |
Calibrating Zero-shot Cross-lingual (Un-)structured Predictions (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing need for model calibration when natural language models are deployed in critical tasks. |
| Approach: | They compare model calibration methods in a context of zero-shot cross-lingual transfer with pre-trained language models. |
| Outcome: | The proposed method fails to calibrate more complex confidence estimations in structured predictions compared to expressive alternatives like Gaussian Process Calibration. |
Tree-Based Representation and Generation of Natural and Mathematical Language (2023.acl-long)
Copied to clipboard
| Challenge: | Existing models for generating and modeling mathematical language are limited . existing models for modeling and generating mathematical language simply treat mathematical expressions as text . |
| Approach: | They propose to combine mathematical expressions and text-based models to generate mathematically valid expressions. |
| Outcome: | The proposed model outperforms baselines on mathematical expression generation tasks. |
Modeling Mathematical Notation Semantics in Academic Papers (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that can predict mathematical notation are unable to analyze mathematical notations reliably. |
| Approach: | They propose two tasks that can be used to train a model that selectively masks notation tokens and encodes left and/or right sentences as context. |
| Outcome: | The proposed model performs better than baseline models trained by masked language modeling compared to baseline models, but is less accurate than token-level models . |
PHEE: A Dataset for Pharmacovigilance Event Extraction from Text (2022.emnlp-main)
Copied to clipboard
Zhaoyue Sun, Jiazheng Li, Gabriele Pergola, Byron Wallace, Bino John, Nigel Greene, Joseph Kim, Yulan He
| Challenge: | Using NLP methods to discover and extract adverse drug events from unstructured textual data is difficult because it requires time-consuming manual curation. |
| Approach: | They propose to use a hierarchical event schema to extract annotated events from medical case reports and biomedical literature to analyze patient data. |
| Outcome: | The proposed dataset is the largest public dataset to date and contains over 5000 events from medical case reports and biomedical literature. |
DIP: Dead code Insertion based Black-box Attack for Programming Language Model (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to attack natural language models are difficult to apply due to the requirements. |
| Approach: | They propose a black-box attack method that generates adversarial examples using dead code insertion. |
| Outcome: | The proposed method outperforms the state-of-the-art black-box attack in both attack efficiency and attack quality on 9 victim downstream-task large code models. |
Accelerating Sparse Matrix Operations in Neural Networks on Graphics Processing Units (P19-1)
Copied to clipboard
| Challenge: | Graphics Processing Units (GPUs) are used to train and evaluate neural networks efficiently. |
| Approach: | They propose two new GPU algorithms for multiplying a matrix by a few-hot vector and fused softmax and top-N selection. |
| Outcome: | The proposed algorithms achieve speedups over state-of-the-art parallel GPU baselines of up to 7x and 50x, respectively. |
How Much Do Robots Understand Rudeness? Challenges in Human-Robot Interaction (2024.lrec-main)
Copied to clipboard
| Challenge: | This paper examines the pressing need to understand and manage inappropriate language within the evolving human-robot interaction landscape. |
| Approach: | They propose to use data cleaning methods to identify inappropriate language in real-time interactions and evaluate natural language models for their proficiency in discerning rudeness. |
| Outcome: | The proposed methods identify and mitigate inappropriate language in real-time interactions and evaluate natural language models for their proficiency in discerning rudeness. |